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Constant energy contour (CEC) of the surface bands in topological insulators varies not only with 
materials but also at different energies. The quasiparticle interference caused by scattering-off from 
defects on the surface of topological insulators is an effective way to reveal the topologies of the CEC 
and can be probed by scanning tunneling microscopy (STM). Using stationary phase approach, a 
general analytic formulation of the local density of states as well as the power laws of the Friedel 
oscillation are present, based only on the time-reversal symmetry and the local geometry around 
the scattering end points on the CEC. Distinct response of surface states to magnetic impurities 
from that of nonmagnetic impurities is predicted in particular, which is proposed to be measured 
in a closed "magnetic wall" setup on the surface of topological insulators. 
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I. INTRODUCTION 

Topological insulators in three dimensions (3D) are 
band insulators which have bulk insulating gap but gap- 
less surface states with odd number of Dirac cones pro- 
tected by the time-reversal symmetry (TRS). As an use- 
ful surface probe, recent angle-resolved photoemission 
spectroscopy (ARPES) experiments demonstrate clearly 
that the bismuth-based class of materials, Bi2X3 (X=Te 
or Se), are 3D strong topological insulators (TI) with a 
single Dirac cone on the surfacoir— . The effective sur- 
face Hamiltonian when close to the Dirac point is given 
by Ho = Tivfz- (a x k), which describes the helical Dirac 
fermions — charge carriers which behave as massless rela- 
tivistic particles with spin locked to its momentum. How- 
ever, compared to the familiar Dirac fermions in par- 
ticle physics, those emergent quasiparticlcs from non- 
trivial surface states of 3DTIs exhibit richer behaviors. 
In Bi2Te3, an unconventional hexagonal warping effect, 
X(k^_+k 3 _)a z , appears due to the crystal symmetry*, un- 
der which the constant energy contour (CEC) of surface 
band evolves from a convex circle to a concave hexagon 
as the Fermi surface climbing up away from the Dirac 
point. This hexagonal warping effect is responsible for 
the observed snowflake shape Fermi surface^, being geo- 
metrically not as simple as a circle, new interesting phe- 
nomena are expected to arise. 

Quasiparticle interference (QPI) caused by scattering- 
off from defects on the surface of 3DTIs is an effective 
way to reveal the topological nature of the surface states. 
The interference between incoming and outgoing waves 
at momenta k^ and kf leads to an amplitude modula- 
tion, Friedel oscillation^, in the local density of state 
(LDOS) at wave vector q = k/ — k^. Nowadays, such 
modulation can be studied by one more powerful sur- 
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face probe, scanning tunneling microscopy (STM), di- 
rectly in real space and provide information in momen- 
tum space through Fourier transform scanning tunneling 
spectroscopy (FT-STS). Several STM measurements^ 
with ordinary (spin-unpolarized) tip have been per- 
formed on the surface of 3DTIs in the presence of non- 
magnetic point and step impurities, and the following fea- 
tures share in common, (i) The topological suppression 
of backward scattering from nonmagnetic point and edge 
impurities is confirmed by the observation of strongly 
damped oscillations in LDOS, companied with the in- 
visibility of the corresponding scattering wave vector q 
in FT-STS. (ii) Anomalous oscillations are reported in 
Bi2Te3 for both point and edge impurities when the 
hexagon warping effect starting to work, (hi) Surface 
bound states exist when near to the point and edge 
impurities. These experimental facts are theoretically 
well-explained by different groups^—, however no gen- 
eral analytic expressions of the LDOS have been pre- 
sented yet. One conclusion in common is now clear that 
for short-range impurities, different from the well-known 
R" 1 and i?" 1 / 2 power laws of Friedel oscillations in two- 
dimensional electron gas (2DEG)^ for point and edge 
impurities, the leading powers of Friedel oscillations in 
helical liquid are dominated by the scattering between 
time-reversal end points (TRP) and are suppressed re- 
spectively to R~ 2 and i? -3 / 2 . This result is the crucial 
reason of the invisibility of the scattering wave vector 
q in FT-STS, and is the direct consequence of the for- 
biddance of backscattering protected by TRS in helical 
liquid. Bearing in mind also the reported anomalously 
pronounced oscillations in LDOS for both point and edge 
impurities^— at bias voltages where the CEC is noncir- 
cular but shapes as a snowflake, we become to realize 
that two ingredients are essential to the oscillations of 
the LDOS, namely, the TRS of the scattering end points 
and the geometry of the CEC. 

Motivated by these arguments, in this work, we present 
a general analytic formulation of the LDOS measure- 
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ment in STM experiments using the stationary phase 
approach^ This approach has been taken successfully to 
study the Ruderman-Kittel-Kasuya-Yosida interaction in 
3D systems with nonspherical Fermi surfaces^, and un- 
usual correlations are obtained. The advantages of this 
approach reside in that it is sensitive only to the local ge- 
ometry around the so-called "stationary points" on the 
CEC, and can be applied not only to bulk states but also 
to surface states. Using this approach, the followings are 
obtained. First, a complete result of the power-expansion 
series of the measured LDOS is tabled for both point- and 
edge-shaped nonmagnetic and magnetic impurities, mod- 
eled by delta potentials, with ordinary and spin-polarized 
STM tip detection. These results depend only on the 
TRS and the local geometry around the scattering end 
points on the CEC at the energy of interest, which ex- 
plain not only the usual and i? -1 / 2 power laws in 
2DEG but also the famous R~ 2 and i?~ 3 / 2 oscillations 
in helical liquid. Second, the most general formulation 
of the LDOS from CEC beyond the hexagon warping ef- 
fects is also present, which plays the role of a first-glace 
guide for future ordinary and spin-polarized STM exper- 
iments on potential TI materials with more complicated 
Fermi surfaces. Furthermore, spin-polarized STM exper- 
iments are focused on in particular. It is found that there 
is no signal for nonmagnetic impurities which is in con- 
sistence with the TRS. While for magnetic impurities, 
pronounced oscillations of LDOS arc predicted irrelevant 
of the TRS of the scattering end points on the CEC, 
which means the backscattcring channels are opened via 
spin-flip processes^ 6 -. A few ARPES experiments with 
magnetic doping are performe d 16 ' 17 , whereas the spin- 
polarized STM ones are still lacking and called on. Fi- 
nally, it is emphasized that our results can also be gen- 
eralized to surface states of TIs in higher dimensions^. 



We mention that to compare with the recent 
experiments 6 - - — in detail, some complexities^ - - — should 
still be involved beyond the delta-potential assumption 
when considering the scattering processes, however this 
is not the main focus of our work. 



The rest of this paper is organized as follows. In Sect. 
HTl we discuss the intuitive picture of the interference 
between helical waves scattered by magnetic impurities 
which break the TRS. In Sect. IIII1 we present the general 
analytic formulation of LDOS for point and edge impuri- 
ties respectively by focusing firstly on those CEC where 
the stationary points are extremal points. We then gen- 
eralize the results to the CEC where the order of the 
first nonzero expansion coefficient around the stationary 
points is greater than two. This work is finally concluded 
in Sect. IT71 




FIG. 1: (Color online) Illustration of charge and spin inter- 
ferences between two counter-propagating helical waves. The 
gray block is a 3DTI with a magnetic edge impurity (green 
stripe) lying in the x-axis on the surface. An incident helical 
wave along y-axis with spin polarized in the ^-direction (blue 
line) is backscattcred by the magnetic edge and the spin is 
flipped (red line). The interference of the two orthogonal he- 
lical waves leads to a constant LDOS in charge channel, but 
a spiral LDOS in spin channel (purple arrows) in j/z-plane. 



II. STANDING WAVE OF THE SPIN 
INTERFERENCE BETWEEN TWO HELICAL 
WAVES 

In the presence of TRS, the backscattcring by nonmag- 
netic impurities on the surface of 3DTIs is known to be 
forbidden ascribed to the obtained 7r Berry's phase dur- 
ing one full rotation of spi n 19 ' 20 . In experiments, this 
manifests in the invisibility of the scattering vector q in 
FT-STS 9 -. It would then be interesting to ask how the 
surface states respond differently to the magnetic impu- 
rities, and what is their characteristic signatures in the 
STM as well as the FT-STS measurements? With mag- 
netic impurities, naively we would expect the interfer- 
ence to be enhanced relative to the nonmagnetic case 
since the backscattering is restored due to the breaking 
of TRS. However, it turns out that the charge interfer- 
ence by magnetic impurities is still suppressed as by the 
nonmagnetic impurities, and there is still lack of notable 
signals in FT-STS with an ordinary STM tip. The bro- 
ken TRS would only manifest itself in the spin channel 
with spin-polarized STM tip21. 

To understand, we first present a simple picture of the 
interference between two counter-propagating orthogo- 
nal helical waves on the surface of a 3DTI, and then 
give a theoretical survey in the next section. Sup- 
pose there is a magnetic edge impurity placed in the 
x-axis on the surface, and a helical wave of the sur- 
face Hamiltonian Hq, ipx = e lkpV ( 1 i ) T /v / 2, where 
the superscript "T" indicates the transpose, is incident 
along y-axis with spin locked to x-direction, see FigfT] 
This wave is then backscattered by the magnetic edge 
and counter-propagates in negative y-direction as ip2 = 
e~ lkFV { 1 —i ) T /\/2, where its spin is flipped to nega- 
tive x-direction. A simple calculation shows that the 
interference between these two counter-propagating he- 
lical waves, %l> = ipx + ?/>2, leads to a constant charge 
LDOS on the surface, but a spiral spin LDOS in yz- 
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FIG. 2: (Color online) Standing wave of spin interference be- 
tween two helical waves inside a closed "magnetic wall" on top 
of a 3DTI surface. The out-of-plane spin LDOS is exhibited 
by the colored rings and the in-plane spin LDOS is indicated 
by the dark arrows. 

plane, (s)^ = [ sm(2kpx) cos(2fc^a;) ], where s = a/2 
is the electron spin operator. This interference pattern of 
two orthogonal helical waves in spin channel can be de- 
tected by spin-resolved STM experiments. To observe the 
standing wave of this spin interference pattern, a setup 
of closed "magnetic wall" as shown in Figgis proposed, 
where a magnetic layer with a hollow hole is deposited 
on top of 3DTI surface. Then inside this hole, a standing 
spin wave is formed with (s y ) ~ sm(2kpR)/ R 1 ^ 2 , (s z ) ~ 
cos(2fci?i?)/i? 1 / 2 , and a loop current = — (2hvp/e)(s z ) 
along the azimuthal direction should be observed. 

In fact, having in mind the transformation properties 
under the time-reversal that O^V © = cr° , Q^ 1 a a & = 
—<j a , and 9 -1 Go(w,k)0 = Gq(oj, — k), where = ia y 
is the time-reversal operator and a = x,y, z, it can 
be proved in the T-matrix formulism immediately that 
the spin LDOS of nonmagnetic impurities vanishes uni- 
formly and the charge LDOS of magnetic impurities is 
identical to that of nonmagnetic impurities, by inter- 
changing k and — k' in the integrals, see Eq.([T]), us- 
ing tr[G (k)<7 a G (k')<r ]_ = -tr[G (-k> a G (-k)a ]. 
Therefore, in the following, we only need to focus on 
the response between nonmagnetic impurity and ordi- 
nary tip as well as that between magnetic impurity and 
spin-polarized tip. 

III. GENERAL FORMULATION OF 
STATIONARY PHASE APPROACH TO QPI ON 
THE SURFACE OF 3DTI 

In this section, we formulate a general result of the 
interference features from quasiparticle scattering on the 
surface of 3DTIs by nonmagnetic and magnetic impuri- 
ties with ordinary and spin-polarized tips using the sta- 
tionary phase approach^. 

A. Point impurity 

We start by considering a point defect. The derivation 
of the LDOS from the constant background measured in 
STM experiments for a single short-range nonmagnetic 
or magnetic impurity with ordinary or spin-polarized tip 
is given by 

x tr [G5(u, k)T"(u,k)G5(u, k>*] .(1) 
In the above, Go(w,k) is in general the free retarded 
Green's function governing the CEC under considera- 



tions, and is that of the topological surface band in par- 
ticular for our interests, where k = (k x , k y ). = V/(l — 
VGq(uj)) is the T-matrix which is k- independent for a 
delta-potential impurity V(r) = VS(r)a^ 1 , and Gq(u) = 

/ T5FF G o( w ' k )- Thc last Pauli matrix in Eq.((T]}, a v , rep- 
resents the polarization of the STM tip, and throughout 
the paper, we use the Greek index fi,is = for nonmag- 
netic impurity or ordinary tip, and fi = a = x,y, z for 
magnetic impurity or spin-polarized tip. 

Without loss of generality, we expand the T-matrix to 
the first order T 1 - 1 * 1 = Va^ and transform the integrand 
in Eq.fll]) to the diagonal basis of the topological surface 
bands, then the measured LDOS becomes 

- 7 5 / w el(k " k '" R 

v S^ m (k,kQS^ m (k,kQ 

where £„ jm (k) are the spin-splitting bands of the sur- 
face states \n, mk), and we have defined £^ m (k, k') = 
(?ik|cr Al |mk / ). To calculate the integrals, it is tricky to 
notice that the contribution to thc LDOS along a given 
direction, say R = Ry (here and thereafter we shall al- 
ways take the y-direction for example) , at large distances 
mainly results in the so-called "stationary points" k^, 
at which 

dky(e,t) _ dk' y (e',t') _ n 

~~dT~ ~ dt> ~ u ' [i) 

where t and t' are respectively some parameter tangent 
to the CEC e and e'. This is because the phase factors 
e ik y (s,t)R an( j gtfc v (e ,t )R var j es rapidly with respect to t 
and t' so that most of thc integrations cancel out except 
at the stationary points. This condition singles out the 
extremal points with nonvanishing second derivatives, 
such as the pairs connected by qi in a convex CEC in 
FigJ3](a) and a concave CEC in FigJ3](b). Moreover, the 
condition © also allows the turning points such as the 
pair connected by c( 2 in Figj3Jb) where the first nonzero 
derivative being of the third order. In the following, we 
will focus only on the extremal points for now and leave 
the general discussions on the others in Sect lHI CI 

Having identified the pairs of stationary points on the 
CEC, the integrals in Eq.@ at large distances are then 
been approximated by the summation of integrals in the 
neighborhood of all the stationary point pairs on the 
CEC. To do this, we first change the integral variables as 
d 2 k = dedk x /fi\vyi\ where v y i = de(k)/hdk y i, and then 
expand the CEC at the extremal points as k y = k V i — 
(k x - k xi ) 2 /2p xi , where p xi = -[d 2 k y {£,k x )/d 2 k xi ]~ 1 is 
the principle radii of the curvature of the CEC at the 
extremal points, which is positive for maxima while neg- 
ative for minima. Under this approximation, Eq.© be- 
comes 




FIG. 3: (Color online) Schematic picture of CEC and station- 
ary points for point and edge impurities, (a) Convex CEC 
where there is only one pair of stationary points connected 
by the red arrow along any given direction for both point and 
line impurities, (b) Concave CEC for point impurity where 
there are multiple pairs of stationary points. Nonstationary 
points are shown for example as blue arrows, (c) Concave 
CEC for edge impurities (brown line) where the slopes (green 
dashed lines) at the pair of stationary points are the same. 
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FIG. 4: (Color online) Fourier transformation of the LDOS 
with R^ 1 and RT 2 power laws. 
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where x = k x — k X i, x' = k' x — k' x j, and all the quan- 
tities at the extremal points (ij) depend still on the 
energies e and e' . The matrix element E^ m (kj,kj) is 
in general some nonzero constant Cf vi (e, e'), except 
when the pair of stationary points are mutually time- 
reversal symmetric |nkj) = Q\mk'j), which leads to 
^ni mj = by symmetry. Examples are shown as the 



pairs of points connected by qi's in FigsG^a) and (b) 
for convex and concave CEC respectively. In such a 
case, by letting \nk) = ( e l6 ' 2 e' 1 ^ 2 )t/ v / 2 and \mk') = 
( e «0 /2 _ e -i8' /2 )t/y / 2 j wheref9 = arctan(fcy/fc x ), weex- 
pand them around the TRP as 8 = 6*.; — a and 8' = 8'j + a' 
where a(a!) = x(x')/kp, then the matrix elements can 
be expanded as a series S° m = m - — (x + x')/2kF + 



[(x + x')/2kp] 3 /3! H — -. Inserting the series into Eq.((4]) and integrating over the energies, we get 
I 



R) 



V 



2ir 2 h 2 R 



1 f e *Wi 
4fc| i? V 



Pa; 7 



\Pxi 



(5) 



which is our desired result of the LDOS in STM measure- 
ments for point impurity. In the above 4>i = — ^sgn(p x ;), 

and we have used dxe iCx2 = y /r TrJ\C\e i ^ sgn ^ and 

dxx 2 e iCx2 = ^J{2\Cf 2 )e-^ s ^ c \ 
There are several comments regarding this result. First 
of all, for a pair of non-TRS stationary points like (\ 2 in 
Fig|3|b), the leading power is given by the first term in 
Eq.©, which is of R~ x . While for a pair of TRS station- 
ary points as qi in Figs[3] (a) and (b), the first nonvan- 
ishing contribution to the power law is dominated by the 
second term in Eq.Q as R~ 2 for nonmagnetic impurity, 
and for magnetic impurity but with ordinary tip. This 
suppression of LDOS is a direct result of the forbiddancc 
of backscattering of helical waves due to TRS. Corre- 
spondingly in FT-STS measurements, there is a sharp 
peak at 2kF for LDOS with R~ l power law, which is 
absent for R~ 2 power law as shown in Fig@] Interest- 
ingly, for magnetic impurities with spin-polarized tip, 
the first term in Eq.© dominates no matter the pair 
of stationary points is TRS or not, and gives the visi- 
bility of the TRS scattering wave vector qi in FT-STS. 
This distinct response of surface states to magnetic from 
that of nonmagnetic impurities provides a crucial crite- 
ria for the breaking of TRS on the surface of TIs^. Sec- 
ond, when integrating over the energies, we have assumed 
v y i , v' y j ^ so that the only poles in the complex energy 
plane are e = e' = u> + id. However, in general, it is pos- 
sible that there are other poles from v y i = or v' y j = 0, 
which means the stationary points in CEC are also the 
extremal points in the energy-momentum dispersion. In 
such a case, we shall further expand v y i (or w' •) around 
u as v y i(e) = v y i(uj) + (dvyi / de)(e — lu) + ■ ■ •, and take the 
first nonzero term to delete the singularities, whereas this 
won't modify the power laws. Finally, note that when 
summation over the stationary point pairs, (ij), we al- 
ways choose one in the pair with positive velocity v y i 
and the other with negative velocity v' y j to obtain the 
retarded Green's function. Using the general result in 
Eq.©, the power laws of LDOS for point impurity are 
summarized in Table. U according to the classification 
of TRS of stationary point pairs on the CEC, which can 
well-explain the recent STM experiments observation in 
3DTI. 

Before going to the experiments, we first use some 
concrete examples to illustrate how the formulae of ([5]) 



TABLE I: Power laws from point impurity 





ordinary 


spin-polarized 


nonmagnetic TRP 


R-' 2 




non-TRP 


R- 1 




magnetic TRP 


R- 2 


R- 1 


non-TRP 


R- 1 


R- 1 



works. The first one is a 2D quadratic CEC, Hq = 
h 2 k 2 /2m, which is isotropic and there are two degen- 
erate spin bands, see Figj5ja). According to our the- 
ory, the main contribution to the LDOS in this example 
comes from the intraband scattering of the same spin ori- 
entation along y-dircction between two extremal points, 
which wc denote as '1' for minimum and '2' for maximum. 
Then at these points explicitly we have k y2 = p x i = k £ , 
k'yi = p'xi = ~ke', k £ = {2me/fi 2 ) 1/2 , v y2 = hk y2 /m, 
v' yl = h,k' yl /m, and Cf 1 = C 22 = 1. Inserting the 
above into Eq.([5]) and keeping only to the leading order, 

we get 6p { ^(oj,Ry) ~ -{Vm 2 /^kp) cos[2k F R)/R, 
which gives power law. Note that the interband 
contribution whereas to the LDOS is from a pair of TRS 
extremal points, which has a R~ 2 power law. In contrast, 
in the example of a 2D Dirac CEC, Ho = 7a • k, there 
is only one non-degenerate band due to the spin split- 
ting, see Figj5Jb). So only intraband scattering between 
a pair of extremal TRP contributes such that C°j mj - =0, 
and the leading power is expected to be R~ 2 . Inserting 
the quantities k y2 = p x2 = e/y, k' yl = p' xl = -e/7, and 

v y i(2) = 7 s g n [ k yi(2)]/fi into Eq.©, we get 6p$(u, Ry) ~ 
(V/ 47r 2 7 2 ) sin(2fci? R)/R 2 , which is the same as our ex- 
pectation. In FigUl the Fourier transformation of the 
LDOS for these two examples is shown, where we see 
that there is a sharp peak at 2/cf for LDOS of the 2DEG 
Hq, which is greatly broadened in the helical liquid Hp- 

In a recent STM measurement of TI, Bi2Te3, with Ag- 
doped point impurities^, clear standing waves and scat- 
tering wave vectors are imaged through FT-STS when 
the Fermi surface is of hexagram shape. It is observed 
that the high intensity regions are always along the F- 
M direction, but the intensity in T-K direction vanishes. 
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This observation can be well-understood using our sta- 
tionary phase theory. Among the three wave vectors q! , 
q2 (or q 2 ) and q3 with high joint density of states as 
shown in FigJ3Jb), two are connected by the stationary 
points, namely qi and q 2 , while q3 (and q2) is not. This 
explains why no standing waves corresponding to q3 are 
observed in FT-STS. Within the other two, stationary 
points connected by qi are also TRP which shall con- 
tribute the power law of R~ 2 according to our result, 
therefore its intensity in FT-STS is too weak to observe 
in the experiment along T-K direction. For wave vec- 
tors q2 and q 2 along T-M direction, q 2 is stationary but 
non-TRS, out result shows that this wave vector has 
power law, which is responsible for the high intensity re- 
ported in Ref.— . 

B. Edge impurities 



x tr [G r (w, k)T"(w, k x )G5(w, k>"] (6) 



where T(u, k x ) = V(l - VG r Q (uj, K))" 1 and G^(u, k x ) = 
J -^-Gq{uj, k). In the presence of edge impurity, we are 
usually interested in the LDOS along the direction per- 
pendicular to the impurity edge. Then the main con- 
tribution to the LDOS now comes from such pairs of 
stationary points where their momentum transfer q = 
ki — kj- is normal to the impurity edge and the "slopes" 
at the pair of points are the same on the CEC, 



-[k y (s,t)-k' y (e',t)]=0. (7) 



Now we turn to the discussion of edge impurity. The 
edge impurity is assumed to orientate in the x-axis on 
top of a 3DTI surface, see FigdJ and be modeled by 
the Hamiltonian V(r) = V8(y)cr tl , The main difference 
of the edge impurity from that of the point impurity is 
the conservation of the momentum along the direction of 
impurity edge, k x , which effectively reduces one of the 
integrations in Eq.([T]), 



Spfiviu, R) 



1 fd 2 kd 2 k' 

IT J (27r) 4 



■(k-k')-R 



This condition allows more possibilities than those im- 
plied by Eq.flHJ for point impurity. One such example is 
shown schematically as qi in FigJ3Jc) where the pair of 



stationary points has the same slope a* 



(the green 



dashed lines), with a, = dk y (e,k X i)/dk X i, which is not 
necessary to be zero. Following the same logic as the 
discussions in point impurity case in the last subsection, 
the CEC is expanded around the stationary points as 
k y = k yi + Oi(k x - k xi ) - (k x - k xi ) 2 /2p xi , and the LDOS 
is approximated by 



Sp$(u>,R) 



V 



EE 

ran ij 



dxe ' 2 Px 



deL 



-ik'-R 



(2tt) 2 uj + i6 - e„ h\v yl \ J (2tt) 2 uj + iS- e' m h\v' - \ 



dx'e 2 "'^ e la " {x - x ' ] 8 x .. 



s~tfl (IV* I ( X X ) 

^ni,mj ni,mj ' ^2 



(8) 



Note that though the requirement of momentum conser- 
vation, a x i(x — x') — 0, makes Eq.® the same as that 
in point impurity case except for a factor 8 X>X ', the def- 
inition of stationary points are physically quite different 



and includes more terms in the summation of stationary 
point pairs (ij). After integrating out k x and the ener- 
gies, it leads to the final result of LDOS in the presence 
of edge impurity 



V 

(2n) 2 Ti 2 V ttR 



EE 



i^r /2 < 



Hkyi — k'AR 



ni,mj ^ni,mj ° 



;'I>, 



k 2 F R 



(9) 



where Pij = PxiP' xj /{p' XJ -Pxi) and = -f sgn(Pjj-). In 
the above we have assumed p X i ^ p' X j , that is we are not 
considering the case where the CEC near the pair of sta- 



tionary points is nested, otherwise the quadratic terms 
in the expansion of CEC cancel out exactly and higher 
orders expansion should be employed. The power laws of 



7 



(a) t (b) (c) 




q = 2fr F q = 2k r q = k F+ + k F _ 



FIG. 5: (Color online) Schematic CEC of (a) quadratic, (b) 
Dirac, and (c) Rashba dispersions. The spin orientations for 
each degenerate band are indicated respectively by the green 
(solid) and purple (dotted) arrows. The stationary points are 
represented by red and blue dots, which are connected by the 
scattering vector q shown as dashed arrows. The intraband 
scattering comes from the stationary points with the same 
color, while the interband scattering comes from those with 
different colors. 

LDOS from edge impurity are summarized in Talbe. [TTl 
which shall be used to explain the recent STM measure- 
ments for edge impurities^. 



TABLE II: Power laws from edge impurity 





ordinary 


spin polarized 


nonmagnetic TRP 


R -3/2 




non-TRP 






magnetic TRP 


R - 3 /2 




non-TRP 


R- 1 ' 2 


R~V 2 



To have a feeling of how Eq.(|9) works, again 
we apply it first to the examples of Hq and 
Hjj used in point impurity. A few lines of cal- 
culations yield that for 2D quadratic dispersion, 
5p$(oj,Ry) = (Vm 2 /27r 2 h' l k 3 F /2 )sm(2k F R - f)/V^fl, 
which recovers the experimental observation in a 
2DEG±1 While for 2D Dirac dispersion, Sp ( ^(u},Ry) = 
(V/8ir 2 j 2 y/^)sm(2k F R+ f)/i? 3/2 , which is a result 
of the absence of backscattering in helical liquid. Similar 
information in reciprocal space can be extracted via FT- 
STS as exhibited in FigUl where a notable sharp peak ap- 
pears at 2k F for a 2DEG, which disappears contrastively 
for helical liquid. 

In the recent experiment by Gomes et ai, a nonmag- 
netic step is imaged by STM topography in clean Sb (111) 
surface with nontrivial topology^. The Fermi surface con- 
sists of one electron pocket at T, which is surrounded by 
six hole pockets in T-M direction, where the surface dis- 
persion shows a Rashba spin splitting. The measured 
LDOS in T-M direction is fitted by a single g-parameter 
using the zeroth-order of Bessel function of the first kind, 
see Fig. 2(c) in Ref.— , which agrees exactly with our pre- 
diction in Table. HU Along T-M direction, the surface 
band can be modeled by a Rashba Hamiltonian where 
the LDOS is dominated by interband scattering between 



a pair of non-TRS stationary points, sec FigfSIc). Ac- 
cording to our analysis, the Friedel oscillation has i?~ 1//2 
power law, which is the asymptotic expansion of Jo(kpR) 
at large distances. Another STM experiment studies the 
edge impurity, however in Bi2Te2, is the work by Alpich- 
shcv et. alX, where a nonmagnetic step defect with 
height about one unit cell is shaped on crystal surface. 
A strongly damped oscillation is reported when the bias 
voltage is at the energy with a convex Fermi surface as 
shown in Fig[3Ja). Though no fitting of the experimen- 
tal data is estimated in this region, our results predict a 
R- 3 / 2 power law. Pronounced oscillations at higher bias 
voltages where the hexagon warping effect emerges are 
observed with R~ l fitting. Despite of the quantitative 
difference with our result of i? -1 / 2 , this oscillations 
have been explained in several other work o 10 ' 11 beyond 
our simple model. 

Finally, it is emphasized that the results in Tables. U 
and [IT] provide a quantitative description of the QPI by 
magnetic impurities in general, and the interference be- 
tween two orthogonal helical waves as discussed in Sect. 
III1 in particular. The interference of helical waves cor- 
responds to the scattering between two TRS stationary 
points, like the qi's in Figs. [3Ja), (b) and (c). The inter- 
esting thing is that the LDOS in charge and spin channels 
from the very same pair of TRS stationary points has 
quite distinct behaviors. With magnetic impurities, the 
power laws of charge LDOS arc R~ 2 and R~ 3 / 2 for point 
and edge impurities respectively, which is a result of TRS 
and have higher power indices than the and i?~ 1//2 
modulations of the spin-polarized LDOS, which mani- 
fests the TRS breaking. So that the charge LDOS decays 
much faster than the spin LDOS, and to tell the different 
response of topological surface states to magnetic impu- 
rities from that of the nonmagnetic ones^, spin-resolved 
STM experiments in the proposed closed "magnetic wall" 
structure in Figf5] are called on in particular. 

C. Friedel oscillations in arbitrarily-shaped CEC 

In this section, we shall complete the most general for- 
mulation of the QPI on the surface of 3DTI by inclusion 
those special cases which have been skipped over in the 
previously discussions. 

For point impurity, under the condition ([3]), we have 
made the extremal points assumption in Sect. IIII Al that 
the expansion of CEC around the stationary points has 
nonvanishing second derivatives. However it is in general 
possible that p X i diverges so that we need to go to the 
third or even higher order expansions. For example, when 
the stationary points are also turning points on the CEC. 
While for edge impurity, under the condition (J7J , besides 
the above example of turning points where on = a'j = 

p~l = p~j — 0, it is also possible that p xl = p' XJ ^ 
but Pij diverges. This happens when the CEC near the 
stationary points is nested, and we need to go beyond 
quadratic term expansions till some power at which the 
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exact nesting property is unbalanced. 



TABLE III: General results of power laws for point impurity 



nonmagnetic TRP R ( ' + ^ 

non-TRP R^^ + ^l 



ordinary spin-polarized 



magnetic TRP R' 



K~> min(l,h) _R _ (T + -H-) 

-TRP R-H+i) R-(r+& 



To understand the LDOS behavior in STM experi- 



ments in these situations, we assume that the first non- 
vanishing terms in the expansions of k y and k' y around 
the stationary points are in general respectively k y = 

kyi + {k x — k x i) 1 and k' y = k' y j + j3j {k' x — k' x j)^, 
where l,h € Z and the /J's are the expansion coefficients 
which is explicitly fif' = [d l k y / dk l xi ) / 1\ and similar for 
(3'j h) . Notice that for edge impurity, if I = h one more 

constrain ftf ^ ' is further required in particular. 
Then similar calculations as performed in Sects. MI Al 
and IIII Bl lead to the following results for point and edge 
impurities separately, that 



J 



V 



Rt 



-Im 



EE 



J(k yi -k' AR 



V 



EE 



e 



XJ I 



l/Cl*ii* 



xj 



^ni,mj ni,mj 



(10) 



(ii) 



where the notations min(l, h) and max(l, h) represent tak- 
ing the minimum or the maximum one in-between I and 
h. The corresponding power laws of LDOS are listed in 
Tables. [0]] and [TV] We see that by setting I = h = 2, 

TABLE IV: General results of power laws for edge impurity 



ordinary spin-polarized 
nonmagnetic TRP R~ ™('.'0 
non-TRP R~ ™**\i,h) 

magnetic TRP R'^WTJ ij"»4w 

non-TRP R~~Z^m R-^tm 



Tables. ITlandlLTlare recovered. Eqs. fp~0|) and f) 1 1 [) in prin- 
ciple can be used to describe the QPI on an arbitrarily- 
shaped CEC, which is very useful as a first-glance guide 
of the freshly cooked experimental data. 

IV. CONCLUSIONS 

In conclusion, general analytic expressions of the 
LDOS are derived in Eqs.©, ©, jTO]), and flTT|) us- 



ing the stationary phase approach, for nonmagnetic and 
magnetic point and edge impurities in ordinary and spin- 
polarized STM experiments. The power laws of Fricdcl 
oscillation are extracted in Tables. Q] to IIVI in particu- 
lar. The QPI from magnetic impurities are focused on, in 
which the interference of charge intensity is indistinguish- 
able from that of nonmagnetic impurities, while the spin 
intensity of magnetic impurities shows distinctive FT- 
STS patterns, which is proposed to be realized in a closed 
"magnetic wall" setup through spin-polarized STM mea- 
surements. Our results depend only on the TRS as well 
as the local geometry around the stationary points on the 
CEC, so that they are suitable especially as a first-glance 
guide of the experiments with ever changing Fermi sur- 
faces, like the emergence of the hexagon warping effects 
in Bi 2 Te 3 . 
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